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Abstract — Facial  recognition  is  one  of  the  most  studied  topics 
in  the  field  of  biometrics  because  of  its  varied  applications. 
Detection  of  dark  colored  faces  and  poorly  illuminated  faces  are 
not  well  studied  in  the  literature  due  to  several  challenges.  The 
most  critical  challenge  is  that  there  is  inadequate  contrast  among 
facial  features.  To  overcome  this  challenge,  a  new  face  detection 
methodology,  which  consists  of  histogram  analysis,  Haar  wavelet 
transformation  and  Adaboost  learning  techniques,  is  proposed. 
The  extended  Yale  Face  Database  B  is  used  to  examine  the 
performance  of  the  proposed  method  and  compared  against 
commonly  used  OpenCV’s  Haar  detection  algorithm.  The 
experimental  results  with  9,883  positive  images  and  10,349 
negative  images  showed  a  considerable  improvement  in  face  hit 
rates  without  a  significant  change  in  false  acceptance  rates. 

Keywords —  face  detection;  illumination;  skin  color  variation; 
Haar-like  features;  OpenCV 

I.  Introduction 

As  computing  power  increases,  biometric  systems  have 
become  an  increasingly  popular  solution  for  security  related 
applications.  Retina  and  fingerprint  scanners  are  relied  upon  to 
accurately  perform  a  wide  range  of  tasks  including 
authentication  of  personnel  to  restricted  sites  and  identification 
of  individual  persons.  Facial  recognition  is  a  rapidly  growing 
area  in  the  field  of  biometrics  with  increasing  potential  because 
of  its  non-contact  nature.  However,  due  to  the  complexities  of 
a  human  face,  detecting  a  face  in  an  image,  the  first  step  to 
perform  facial  recognition,  is  by  no  means  a  simple  task. 

Many  methods  exist  to  address  this  issue,  such  as  Principal 
Component  Analysis  (PCA),  Hidden  Markov  Models  (HMM), 
and  Haar-like  features.  Viola  and  Jones  proposed  a  method 
using  Haar-like  features  to  rapidly  detect  faces  within  an  image 
[1].  The  features  are  calculated  based  on  the  changes  in 
contrast  values  between  adjacent  rectangular  groups  of  image 
pixels,  but  not  the  intensity  value  of  a  single  pixel.  However, 
when  the  brightness  intensities  of  an  image  are  very  similar, 
and  show  little  variation,  the  detection  algorithm  is  unable  to 
accurately  detect  faces.  This  is  often  the  case  in  poorly  lit 
images  or  in  people  who  have  darker  skin  tones  As 

indicated  in  [5],  “it  is  an  open  secret  that  the  performance  of 
current  face  detection  and  expression  recognition  systems  tends 
to  be  much  lower  when  applied  to  individuals  with  dark  skin”. 
This  is  no  exception  for  the  widely  used  Haar  detection  method 
as  it  relies  on  the  contrast  between  facial  features,  such  as 
forehead-to-eyes,  eyes-to-nose,  etc. 


In  this  paper,  we  present  a  new  method  for  face  detection. 
The  objective  is  to  improve  the  performance  in  detecting  dark 
coloured  faces  or  faces  captured  in  suboptimal  lighting 
conditions.  We  examined  the  skin  color  properties  in  several 
common  color  spaces  such  as  RGB,  Normalized  RGB,  YCbCr, 
and  HSV,  investigated  different  image  enhancement  techniques 
[6-8],  and  developed  an  image  transformation  algorithm.  The 
efforts  were  focused  on  rescaling  face  images  with  increased 
contrast  and  therefore  making  them  detectable  by  the  Haar  face 
detection  method.  The  experiments  were  carried  out  on  both 
color  and  grey  scale  images,  which  demonstrated  significant 
improvements  in  terms  of  face  detection  hit  rates.  The  rest  of 
the  paper  is  organized  as  follows.  Haar-like  features,  integral 
image,  and  AdaBoost  learning  are  described  in  Section  2.  The 
proposed  approach  is  presented  in  Section  3.  Experimental 
results  are  presented  in  Section  4  and  the  paper  is  concluded  in 
Section  5. 

II.  Haar-Like  Features 

A.  Introduction  to  Haar-like  features 

Object  detection  is  performed  by  identifying  intensity 
patterns  common  to  a  positive  image  set.  When  run  against  a 
test  sample,  a  positive  match  is  found  if  the  image  possesses 
enough  of  the  same  features. 

Haar-like  features  have  been  widely  used  in  different 
boosting  algorithms  and  object  detection,  especially  face 
detection  [1].  The  method  contains  three  main  ideas  to  achieve 
real-time  object  detection  performance:  integral  images  for 
fast  computation,  AdaBoost  for  feature  selection,  and 
classifier  cascades  for  fast  rejection  of  non-face  windows. 

An  integral  image  is  created  to  quickly  calculate  the  sum  of 
intensity  values  in  a  rectangle  subset  image.  It  is  a 
representation  of  the  original  image  where  the  resulting  value 
at  any  location  is  the  sum  of  all  the  intensity  values  contained 
within  the  rectangle  formed  by  the  x  and  y  coordinates  of  the 
original  image.  Figure  1  shows  an  example  in  which  integral 
image  value  658  equals  to  122  +  245  +  215  +  76  in  the  subset 
image. 
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Fig.  1 .  Conversion  of  greyscale  values  to  integral  image 
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Sum  of  Intensities  =  D-  C-  B  +  A 


Fig.  2.  Method  of  calculating  intensities  with  the  integral  image 

The  integral  image  is  useful  because  it  allows  for  the  sum 
of  intensity  within  a  rectangle  of  any  size  to  be  calculated  in 
constant  time  through  the  formula  shown  in  Figure  2.  As 
Haar-like  features  focus  on  differences  in  the  average 
intensities  between  object  rectangular  image  subsets,  integral 
image  is  a  valuable  representation  for  fast  calculation. 

An  AdaBoost-based  algorithm  is  used  to  select  features  that 
are  used  for  facial  classification.  This  algorithm  functions  by 
weighting  weak  features  (slightly  better  than  random  selection) 
and  combining  a  collection  of  them  to  form  a  stronger  and 
more  reliable  classifier.  Each  classifier  is  a  set  of  patterns 
commonly  found  in  faces.  A  Haar  cascade  is  a  collection  of 
these  pattern  groupings.  AdaBoost-like  algorithm  is  a  training 
process  that  combines  weak  features  into  a  strong  classifier  to 
maximize  the  hit  rate  for  a  positive  set  of  images.  A  Haar 
cascade  is  trained  on  a  collection  of  images  demonstrating  the 
properties  which  the  user  wishes  the  cascade  to  look  for,  and  a 
second  group  of  images  which  do  not  contain  the  desired 
information.  AdaBoost  takes  both  image  collections  and 
adjusts  the  classifiers  to  match  the  positive  images  as  closely  as 
possible  while  excluding  patterns  that  match  the  negative 
images.  The  cascade  is  composed  of  several  stages  each  of 
which  filters  out  different  non-faces.  These  stages  are  arranged 
in  such  a  way  that  the  more  complex  the  stages  are,  the  later 
the  operations  will  be.  This  makes  fewer  images  be  processed 
by  complex  stages  to  increase  computational  efficiency.  This 
multistage  process  is  both  fast  and  accurate  with  a  reported 
detection  rate  as  high  as  15  frames  per  second  [1]. 

B.  Issues  with  Haar-like  features 

It  has  been  mentioned  that  low  illumination  and  varying 
brightness  have  a  negative  effect  on  various  face  detection 
methods  [2-4].  This  issue  also  factors  into  Haar  face  detection 
as  faces  that  are  in  unconstrained  lighting  conditions  return 
inconsistent  results.  A  similar  concern  is  that  people  with 
darker  skin  also  tend  to  display  fewer  differences  in  the  light 
patterns  on  their  skin  [2],  making  it  difficult  for  specific  facial 
features  to  be  matched.  When  there  is  little  difference  in  the 
intensity  of  the  image,  it  becomes  problematic  for  the  Haar 
feature  detection  algorithm  to  match  patterns  to  faces.  In  these 


circumstances,  the  algorithm  will  often  fail  to  return  favourable 
results. 

In  the  literature,  one  common  approach  to  address  this 
problem  is  histogram  equalization  [8-10].  Histogram 
equalization  is  a  method  that  creates  an  image  in  which  the 
number  of  intensity  values  is  evenly  distributed.  This  is 
accomplished  by  adjusting  the  intensity  of  the  image  based  on 
the  occurrences  of  each  distinct  intensity  value. 

The  drawback  of  global  histogram  equalization  is  that  an 
image  with  faces  of  different  skin  tones  ranging  from  dark  to 
fair  never  shows  such  a  distribution.  Another  major  issue 
occurs  when  a  strong  bias  towards  one  end  of  the  spectrum 
exists  (such  as  a  well-illuminated  face  on  a  dark  background). 
In  such  a  case,  the  image  produced  by  histogram  equalization 
appears  quite  unnatural  [11]. 

III.  Proposed  Method 

Due  to  the  low  detection  rate  of  using  Haar-like  features  on 
faces  under  poor  or  irregular  lighting  conditions,  we  have 
developed  a  new  algorithm  based  on  the  histogram 
specification  technique  to  improve  the  appearance  of  extra  dark 
and  extra  bright  faces.  As  mentioned  above,  there  is  difficulty 
in  detecting  dim  or  dark-skinned  faces  due  to  the  small 
difference  in  intensity  values.  Therefore,  a  non-linear 
transformation  algorithm  is  proposed  that  uses  a  decreasing 
exponential  function  to  increase  variation  of  low  intensity 
values  while  preserving  high  intensity  areas. 

First,  the  image  is  converted  to  the  hue,  saturation,  and 
value  colour  scheme  (HSV).  The  algorithm  is  then  applied  to 
the  value  channel.  Our  algorithm  alters  the  value  raising  the 
spread  and  intensities  of  dark  pixels  while  maintaining  the 
difference  at  the  midrange.  The  new  image  is  converted  to 
grayscale  and  the  Haar  detector  is  used  to  search  for  faces. 

Different  from  histogram  equalization,  the  proposed 
method  provides  a  non-linear  transformation  on  the  brightness 
of  pixels  allowing  a  detectable  difference  in  intensity  values  for 
Haar-like  features.  In  addition,  it  does  not  depend  on  the  rest 
of  the  image  assuring  that  the  brightness  values  of  a  face  are 
always  shifted  in  the  right  direction. 

Figure  3(a)  shows  an  example  of  a  face  image  captured 
under  low  illumination  condition.  Histogram  equalization  is 
performed  and  the  resulting  image  and  corresponding 
histogram  are  shown  in  Figures  3(c)  and  3(d),  respectively. 
Since  many  pixels  fall  in  the  low  intensity  range,  the  histogram 
equalized  image  has  unevenly  spread  brightness  values. 
Although  the  low  intensity  range  has  been  greatly  increased, 
the  high  intensity  range  is  narrowed  down,  which  decreases  the 
image  quality  and  may  make  it  difficult  to  detect  Haar-like 
features.  From  the  resulting  histogram  generated  by  the 
proposed  method,  it  can  be  seen  that  the  spread  of  low  intensity 
pixels  has  also  been  increased,  and  the  high  intensity  areas  are 
not  compressed  (Fig.  3(f)).  Therefore,  both  dark  facial  features 
and  bright  facial  features  are  mostly  preserved  (Fig.  3(e)).  A 
number  of  experiments  have  been  carried  out  to  determine  the 
accuracy  and  advantages  of  the  proposed  method. 


(c)  Resulting  image  of  histogram  equalization  (d)  Equalized  histogram 


(e)  Resulting  image  of  proposed  method  (f)  Specified  histogram 


Fig.  3.  Histograms  of  a  sample  face  (top:  low  illumination,  middle:  histogram  equalized,  bottom:  proposed  method) 


(a)  Faces  detected  by  original  Haar  algorithm 


(b)  Faces  detected  by  the  proposed  method 


Fig.  4.  Comparison  of  Haar  detector  to  proposed  method 


IV.  Experimental  Results 

In  order  to  evaluate  the  performance  of  the  proposed 
approach,  several  experiments  have  been  conducted  with  both 
color  and  greyscale  images.  In  the  test,  two  measurements,  hit 
rate  and  false  acceptance  rate  (FAR)  are  used.  The  hit  rate  is 
defined  as  the  total  number  of  detected  faces  over  the  total 
number  of  faces  in  the  test  images.  While  the  FAR  is  defined 
as  the  total  number  of  non- face  objects  that  are  mis-detected  as 
faces  over  the  total  number  of  faces  in  the  test  images. 

First,  we  collect  a  set  of  images  of  human  faces  from 
Internet  [12,  13]  to  test  the  ability  of  the  proposed  approach  in 
detecting  dark-skinned  faces.  Figure  4  presents  an  example  that 
demonstrates  the  algorithm’s  improvement  to  Haar-like 
features  detection.  Since  they  have  small  intensity  differences 
between  darker  areas  (e.g.  eyes,  nostrils)  and  skin,  some  faces 


cannot  be  detected  by  original  Haar  algorithm  (Fig.  4(a)).  The 
proposed  algorithm  addresses  this  issue  by  increasing  the  range 
of  low  intensity  pixels.  Therefore,  the  hit  rate  on  dark  coloured 
faces  has  considerably  increased  (Fig.  4(b)).  Figure  5  shows  a 
more  difficult  example  of  detecting  half-shadowed  faces.  The 
Haar  detector  picked  up  one  of  the  half-shadowed  faces  (Fig. 
5(a)),  while  the  proposed  approach  detected  both  half- 
shadowed  faces  successfully  (Fig.  5  (b)). 

To  further  evaluate  the  proposed  approach,  a  comparison 
test  is  carried  out  that  compares  our  face  detection  result  to  the 
results  obtained  from  original  Haar  face  detection  algorithm 
provided  by  OpenCV  and  an  algorithm  which  used  histogram 
equalization.  The  test  is  performed  against  17  subjects  under 
576  viewing  conditions  from  the  extended  Yale  Face  Database 
B.  Figure  6  shows  a  comparison  of  image  enhancement  and 
face  detection  results  between  histogram  equalization  and  the 
proposed  algorithm.  It  can  be  seen  that  histogram  equalization 
and  the  proposed  method  are  both  able  to  restore  images  from 


(a)  Detection  result  by  original  Haar  algorithm 


(b)  Detection  result  by  the  proposed  method 


Fig.  5.  Example  of  detecting  faces  under  shadows 


V.  Conclusions 


In  this  paper,  we  have  proposed  a  method  to  improve  the 
performance  of  Haar  face  detector  in  OpenCV.  The 
experiments  demonstrate  that  the  proposed  method  for  face 
detection  using  Haar-like  features  by  way  of  illumination 
correction  presents  superior  results  than  the  unaltered 
algorithm.  The  original  Haar  face  detection  method  has 
difficulty  in  dealing  with  dark  coloured  faces  or  faces  captured 
under  poor  lighting  conditions.  While  histogram  equalization 
is  a  commonly  used  method  to  improve  these  results,  it  is 
prone  to  overcompensation  and  the  results  are  highly 
dependent  on  the  background.  The  proposed  transformation 
algorithm  addresses  this  issue  by  increasing  the  spread  of  low 
intensity  pixels  while  preserving  the  mid-  and  high-range 
areas.  The  returned  image  appears  to  be  under  a  more 
consistent  lighting  and  is  more  distinguishable.  Experimental 
results  from  the  proposed  method  prove  to  be  an  effective  way 
of  dealing  with  this  issue  with  a  considerable  increase  in  hit 
rate  without  severely  impacting  the  false  acceptance  rate. 
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(c)  Images  obtained  by  the  proposed  method 

Fig.  6.  Experiment  results  on  the  Yale  B  database. 

dark  conditions.  However,  the  inconsistency  of  histogram 
equalization  can  be  seen  by  the  wide  range  of  intensity  values 
on  the  face.  In  Figure  6(b),  due  to  the  different  background 
conditions,  subject  5  (counting  from  left  to  right)  appears 
brighter  and  the  features  on  the  left  side  of  the  face  are 
indistinguishable.  Therefore,  subject  5  could  be  detected  by 
the  proposed  algorithm  (Fig.  6(c)),  but  not  through  the 
histogram  equalized  image  Fig.  6(b).  Comparing  subject  4  in 
Figures  6(b)  and  6(c),  the  image  quality  when  enhanced  by 
histogram  equalization  is  poorer  than  that  of  the  proposed 
method.  The  eyes  are  better  defined  and  that  the  overall 
picture  appears  to  be  under  a  more  consistent  lighting. 


TABLE  I.  Comparison  of  Face  Detection  Results 


Algorithm 

#Positives 

#Nagatives 

Hit  Rate  (%) 

FAR  (%) 

Basic  OpenCV 
HAAR 

7471 

2608 

75.6 

25.2 

Histogram 

Equalization 

9255 

2772 

93.6 

26.9 

Proposed 

Approach 

9615 

2646 

97.3 

25.6 

Table  1  shows  a  comparison  of  test  results.  Of  the  three 
methods  tested,  the  original  OpenCV  algorithm  proved  the 
least  accurate,  having  a  hit  rate  of  only  75.6%.  It  also  had  the 
lowest  FAR  but  only  by  a  slight  margin  at  25.2%.  The 
histogram  equalized  images  had  the  second-best  positive 
acceptance  rate  of  93.6%  and  the  worst  FAR  at  26.9%.  The 
new  method  proposed  in  this  paper  outperforms  the  other  two 
algorithms  with  a  positive  acceptance  rate  of  97.3%  and  a  FAR 
only  slightly  worse  than  that  of  the  original  OpenCV  Haar 
function  at  25.6%.  The  images  used  in  false  acceptance  test 
were  a  compilation  composed  of  cows,  cars,  bikes,  buildings, 
and  toys. 
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